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DETAILED ACTION 

The response received on 3/08/200 has been placed in the file and was 
considered by the examiner. An action on the merit follows. 

Response to Amendment 

1 . The amendments filed on 08 March 2007 have been fully considered. Response 
to these amendments is provided below. 

Summary of Arguments/Amendments and Examiner's Response: 

2. The applicant has cancelled all claims and added new claims. The new 
independent claims are similar to the cancelled independent claims with added 
limitations including "wherein the final target object location is given by a weighted mean 
of all lopation hypotheses of said 3D target location hypotheses for combining the 
measurements, and whereby the examples of said target object can comprise human 
appendage, human hand, and human head," and "whereby examples of imaging 
sensors can comprise color cameras and IEEE 1394 cameras." 

3. Applicant's arguments with respect to the claim have been considered but are 
moot in view of the new ground(s) of rejection. 

4. The following comments are regarding the pro se status of the application: 

5. Applicant should submit an argument under the heading "Remarks" pointing out 
disagreements with the examiner's contentions. Applicant must also discuss the 
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references applied against the claims, explaining how the claims avoid the references or 
distinguish from them. 

6. The applicant has argued a few things about the claims that overcome the prior 
art. However, the applicant has provided extra arguments, which leads the examiner to 
provide a few comments. The examiner would like to remind the applicant that in order 
for an application to be allowable, the independent claim must be allowable. The 
dependent claims are then also allowable. If the applicant believes that the 
independent claim is allowable over the prior art by the necessitated amendment and is 
correct, it is not necessary to argue the dependent claims. For example, the applicant 
argues that "Random distribution of the hypotheses" is not in the prior art as well as 
"combination of spread out color..." If this is thought to be the allowable feature, it 
should be in the independent claim. Also, the examiner would like to remind the 
applicant that it is the examiner's responsibility to give the claims the broadest possible 
interpretation. In the remarks, the applicant argues many points that are provided in the 
specification. The examiner does not have to reject the specification, only the content 
of the claims, in its broadest interpretation. It is more effective to argue that a specific 
part of the claim is missing, not just the ideas that are represented in the claims. Also, 
some arguments, such as the reference not disclosing the same solution as the 
applicant, are not important as well. As long as the reference teaches what is in the 
claims, the reference is valid. The reference's specification can explain different 
reasons for doing what is claimed, but the fact that it does the same thing is enough. 
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7. Also, the examiner would like to remind the applicant that the preamble to the 
claim is not given any patentable weight because it doesn't breath life or vitality into the 
claim. 

8. The following is a standard paragraph provided for pro se applicants: An 
examination of this application reveals that applicant is unfamiliar with patent 
prosecution procedure. While an inventor may prosecute the application, lack of skill in 
this field usually acts as a liability in affording the maximum protection for the invention 
disclosed. Applicant is advised to secure the services of a registered patent attorney or 
agent to prosecute the application, since the value of a patent is largely dependent upon 
skilled preparation and prosecution. The Office cannot aid in selecting an attorney or 
agent. 

A listing of registered patent attorneys and agents is available on the USPTO 
Internet web site http://www.uspto.gov in the Site Index under "Attorney and Agent 
Roster." Applicants may also obtain a list of registered patent attorneys and agents 
located in their area by writing to the Mail Stop OED, Director of the U. S. Patent and 
Trademark Office, PO Box 1450, Alexandria, VA 22313-1450 



Specification 

9. The amendment filed 3/26/2007 is objected to under 35 U.S.C. 132(a) because it 
introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment 
shall introduce new matter into the disclosure of the invention. The added material 
which is not supported by the original disclosure is as follows: Claims 26 and 36 claim a, 
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"means for adding random displacements to the location hypothesis at each time step 
for maintaining 3D target location hypotheses." The applicant has pointed out a place in 
the specification that partially matches this claim when stating, "new hypotheses are 
created from the hypotheses as the previous time step t-dt by randomly selecting 
previous hypotheses..." This is not the same as what is claimed. Previous hypotheses 
are randomly selected, which is not the same as adding random displacements. If what 
is stated in the claim is supported somewhere else is the specification, the examiner 
with withdraw this objection. 

Applicant is required to cancel the new matter in the reply to this Office Action. 

Claim Objections 

10. Claims 17 and 27 are objected to because of the following informalities: claims 
17 and 27 have odd wording that makes it difficult to interpret the claim. The claims 
state, "wherein the final target object location is given by the weighted mean of all 
location hypotheses of said 3D target location hypotheses for combining the 
measurements." The addition of "for combining the measurements" is confusing, since 
it is placed in an odd space. It could read that the 3D target location hypotheses are 
there for combining the measurements, or it could read that any other of the objects in 
the line are for combining the measurements. It is also unclear as to what the 
measurements are; if they are the measured confidences of step e, or other 
measurements that are not mentioned in the claim that are combined, or if they are 
measurements in the claim that are combined but not previously mentioned as 
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combined. Appropriate correction is required. The claim is given its best interpretation 
and is rejected below. 

Claim Rejections - 35 USC §112 

11. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

12. Claims 26 and 36 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. Claims 26 and 36 contain claimed subject 
matter that does not match the specification, as explained above in the objection for the 
specification. If the applicant can point out a place in the specification where the 
wording is consistent/supported with the wording in the claim, the rejection will be 
withdrawn. 

13. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

14. Claims 17-36 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 
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15. Regarding claims 17 and 27, the phrase "examples... can" renders the claim 
indefinite because it is unclear whether the limitation(s) following the phrase are part of 
the claimed invention. See MPEP § 2173.05(d). The statement of having something 
that "can" be included in the claim does not necessarily mean that it is part of the 
invention and does not add any limitation to the claims and isn't given any patentable 
weight. 

16. Claims 17 and 27 recite the limitation "the final target object location" in lines 1 1 
and 10, respectively. There is insufficient antecedent basis for this limitation in the 
claim. 

17. Claims 17 and 27 recite the limitation "the weighted mean" in lines 1 1 and 10, 
respectively. There is insufficient antecedent basis for this limitation in the claim. 

18. Claims 17 and 27 recite the limitation "the measurements" in lines 12 and 11, 
respectively. There is insufficient antecedent basis for this limitation in the claim. 

19. Claims 19 and 29 recite the limitation "the target object location" in lines 4 and 4, 
respectively. There is insufficient antecedent basis for this limitation in the claim. It is 
unclear if this is referring to the same final target object location as disclosed in claims 
17 and 27. 

20. Claims 19 and 29 recite the limitation "the confidences region" in lines 4 and 4, 
respectively. There is insufficient antecedent basis for this limitation in the claim. 

21. Claims 19 and 29 recite the limitation "the central location" in lines 4 and 4, 
respectively. There is insufficient antecedent basis for this limitation in the claim. 
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22. Claims 20 and 30 recite the limitation "the combination of the spread out images" 
in lines 5 and 6, respectively. There is insufficient antecedent basis for this limitation in 
the claim. It is unclear if that "the combination of the spread out images" is the same as 
"the combination of color and motion cues," but as it reads, it is different and there is no 
antecedent basis. Furthermore, "the spread out images" is not defined earlier in the 
claim either; only color and motion cues are spread out. 

23. Claims 20 and 30 recite the limitation "the edges" in lines 6 and 7, respectively. 
There is insufficient antecedent basis for this limitation in the claim. 

24. Claims 20 and 30 recite the limitation "the interior" in lines 7 and 8, respectively. 
There is insufficient antecedent basis for this limitation in the claim. 

25. Claims 23 and 33 recite the limitation "the new hypotheses" in lines 4 and 4, 
respectively. There is insufficient antecedent basis for this limitation in the claim. 

26. Claims 23 and 33 recite the limitation "the weights" in lines 4 and 4, respectively. 
There is insufficient antecedent basis for this limitation in the claim. It is unclear if this 
corresponds to the weighted average or if this is another weight. 

27. Claims 25 and 35 recite the limitation "the space" in lines 2 and 2, respectively. 
There is insufficient antecedent basis for this limitation in the claim. 

28. The examiner would also like to suggest that the applicant amend the claims to 
read so that all "location hypotheses" that correspond to the "3D target location 
hypotheses" to read the same. The original claims read much clearer regarding the 
target location hypotheses because there is only one "target location hypotheses," the 
3D target location hypotheses. With the added amendments, the claim is confusing 
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because it mentions many variations of the same theme, and it is unclear if this is the 
same since sometimes the applicant adds a "the" in front, and sometimes the applicant 
does not. This can be seen in all of the steps of the independent claim, for instance: 

- Step b: "3D target location hypotheses," 

- Step c : u each target location hypothesis" = step b's hypotheses? 

(could amend to read, "each of the 3D target location hypotheses") 

- Step e "the hypotheses" = step b's hypotheses? = step c's hypotheses? 

- Wherein statement: "all location hypotheses" = step b's? = step c's?=step e's?...etc. 

This can be further seen in other dependent claims such as claims 19 and 29 where it 
states "the target object location," which should correspond to "the final target object 
location" of the independent claim, and would be clearer if amended to be consistent 
with the independent claim. Please look through claims and amend the terminology to 
be consistent. If a term is intentionally different, please amend these terms to clearly be 
different. 

29. Regarding claims 18-20, 23, 28-30 and 33, the claims claim "means for 
projecting," "means for measuring overall confidence," "means for measuring said 
confidences," and "means for maintaining 3D target location." It is unclear if these 
means correspond to the means claimed in the corresponding independent claim. 

30. Regarding claims 23 and 33, the claim claims 3D target location hypotheses 
without a "the" or "said" in front of it, as seen in "for maintaining 3D target location 
hypothesis by creating a set of 3D target location hypotheses..." By not including a 
"the" or a "said", this infers that the 3D target location hypotheses in claims 23 and 33 
are different than those of claims 17 and 27, which the examiner infers is not the 
applicants intentions. The same problem occurs in claims 24, 26, 34 and 36. 
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Claim Rejections - 35 USC § 103 

31 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

32. Claims 17-36, as best understood, are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent No. 6999624 (Nelson) in view of U.S. Patent Application 
Publication No. 20010008561 (Paul et al), and further in view of U.S. Patent No. 
5323470 (Kara et al). 

Regarding claim 27, Nelson discloses an apparatus comprising: a) means for 
capturing the target object, the means that obtains the digital/ digitized image (fig. 1, 
item 100), b) means for maintaining a large number of 3D target location hypotheses, 
the preprocessing section (fig. 1, item 102) that maintains many locations in which the 
target can be found (fig. 2), c) means for projecting each target location hypothesis from 
3D space to 2D image spaces of means for capturing the target object, done also by the 
preprocessing section (fig. 1, item 102), since the image obtained is a 2D image from 
the means for capturing the target object, obtained from a 3D space and thus projected 
into 2D by the means for capturing the target object, and all the location hypotheses are 
found as divided in fig. 2, d) means for measuring confidences about the presence of 
the target object in images captured by the means for capturing the target object (fig. 1 , 
item 12) in which confidences/ scores are found in each window/ hypotheses (fig. 3, 
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item 24) and e) means for combining the measured confidences of the hypotheses to 
obtain 3D location of the target object (fig. 3, item 38), since those with a high enough 
measured confidence are combined/ grouped to obtain the 3D location since the 
location is found using the 3D hypotheses stated above and also because the location 
is found of the 3D object, wherein the final target object location is given by the 
weighted mean of all location hypotheses of said 3D target location hypotheses for 
combining the measurements, since originally a weighted sum that is normalized (col. 2, 
lines 39-43), and thus a weighted mean is found in each window, and is thus used to 
give the final target location of all the hypotheses (col. 2, lines 64-69) and also used for 
combining the measurements (col. 3, lines 58-60). 

Nelson does not disclose expressly that hypotheses are in areas that are 
guessed to have the object based on other information, thus narrowing the search area, 
and the means for capturing the target object has two or more imaging sensors. 

Paul et al discloses that in an image, probable locations can be found to find the 
hypotheses (page 1, paragraph 10). 

Nelson and Paul et al are combinable because they are from the same field of 
endeavor, i.e. image tracking. 

At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to have narrowed-down hypotheses. 

The suggestion/motivation for doing so would have been to have a faster system 
by only processing areas of interest instead of processing the full image. 
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Nelson (as modified by Paul et al) does not disclose expressly that the means for 
capturing the target object has two or more imaging sensors. 

Kara et al discloses that a means for capturing a target object has two imaging 
sensors, 2 CCD cameras shown in fig. 1. 

Nelson (as modified by Paul et al) and Kara et al are combinable because they 
are from the same field of endeavor, i.e. image tracking. 

At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to have plural cameras. 

The suggestion/motivation for doing so would have been to provide a more 
accurate system by providing many possible images, and to provide a faster system by 
providing multiple images at the same time 

Therefore, it would have been obvious to combine the apparatus of Nelson with 
the hypotheses of Paul et al and the plural cameras of Kara et al to obtain the invention 
as specified in claim 27. 

(It is reminded that the statement of having something that "can" be included in 
the claim does not necessarily mean that it is part of the invention and does not add any 
limitation to the claims and isn't given any patentable weight.) 
33. Claim 17 is rejected for the same reasons as claim 27. Thus, the arguments 
analogous to that presented above for claim 27 are equally applicable to claim 17. 
Claim 17 distinguishes from claim 27 only in that claim 17 is a method claim and claim 
27 is an apparatus claim. Since an apparatus carries out a method, prior art applies. 
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34. Regarding claim 28, Paul et al discloses projecting target locations, or predicting 
target locations, with projections that are obtained by calibrating said imaging sensors 
with respect to a reference coordinate system, the reference coordinate system being 
the system shown at the point of initialization, and where all future tracking is based on 
(page. 2, paragraph 0020). 

35. Regarding claim 29, Nelson discloses that the apparatus further comprises 
means for measuring overall confidence of the target object location based on a 
combination of a plurality of confidence images (fig. 3, step 42), since the overall 
confidence is calculated based on a group image, the confidence images being the 
images that are scored in fig. 3, item 24, from each of the imaging sensors, since this is 
done for an image from a sensor, and thus for each sensor, wherein the target object 
location is given as the central location of the confidences region, interpreted as the 
combination region, since the target location is given as the full group including the 
central locations of the group (fig. 3, item 44). 

36. Regarding claim 30, Nelson discloses that the means for measuring confidences 
measures confidences are based on color (col. 3, line 52- col. 4, line 5). Paul et al also 
discloses means for measuring confidences, the means that measures the probability of 
the target center and the probable location (page 2, paragraph 30), page 3, paragraph 
39 and 40) and thus finding likely areas, based on a combination of color and motion 
cues in each of the images captured by said imaging sensors (page 1, paragraph 10), 
wherein the color and motion cues are spatially spread out using averaging, since the 
color cues are spread out using a weighted average, spreading out the applicable area 
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to pixels in a given region (page 2, paragraph 30) and the motion cues being spatially 
spread out since the motion cues utilize the spatially spread out color cues (page 3, 
paragraph 39-40), before they are combined since the spreading out occurs before the 
motion and color cues are combined, and whereby the combination of the spread out 
images results in an image where a combined response is not only present at some of 
the edges of the target object but also contains strong responses in the interior of the 
target object, since the edges of the target are considered (page 2, paragraph 30), and 
the central area is defined (page 3, paragraph 40). 

37. Regarding claim 31, Paul et al discloses means for calculating the color cues, or 
the colors that are detected as likely colors, using a color model of the target object, a 
reference (page 2, paragraph 27-28), wherein the color model of the target object is 
represented by something that is estimated by collecting color samples of the target 
object, since a reference is found by taking the ideal sample color of the target (page 2, 
paragraph 0020). Paul et al does not expressly disclose that the "something" that 
represents a color model is a histogram. Nelson discloses that a histogram can be 
used to represent color (col. 3, lines 60-66). 

38. Regarding claim 32, Paul et al discloses that motion cues are calculated by 
measuring differences between images captured sequentially by said imaging sensors 
(page 3, paragraph 38). 

39. Regarding claim 33, Paul et al discloses that 3D target location hypothesis are 
maintained by creating a set of 3D target location hypotheses at each time step, since 
at each time step, when detecting motion, Paul et al creates a new set of hypotheses 
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based on the motion (page. 3, paragraph 36-38), wherein the weights, (being any 
weight since there is not antecedent basis, therefore, the weights can be the 
confidences) of the new hypotheses are given by said confidences, the calculations 
made for the highest probability (page 2, paragraph 30 and page 3, paragraph 40). 

40. Regarding claim 34, Nelson discloses means for creating said 3D target location 
hypotheses, that which maintains the 3D target location hypotheses, as disclosed in the 
independent claim. Paul et al discloses that hypotheses can be based on known 3D 
target location hypotheses from a previous time step (page 3, paragraphs 36 and 38). 

41. Regarding claim 35, Nelson discloses means for initially distributing the 3D target 
location hypotheses randomly in the space viewed by said imaging sensors by dividing 
the space into many block (fig. 2) and thus providing random 3D target location 
hypotheses based on no other previous knowledge of the target location. 

42. Regarding claim 36, Nelson discloses means for adding random displacements 
to the location hypothesis at each time step for maintaining 3D target location 
hypotheses since Nelson shows many location hypotheses shown in fig. 2, and any 
random displacement can be added from any of the windows, and another window will 
be another 3D target location hypotheses. 

43. Claims 18- 26 are rejected for the same reasons as claims 28-36. Thus, the 
arguments analogous to that presented above for claims 28-36 are equally applicable to 
claims 18-26. Claims 18-26 distinguish from claims 28-36 only in that they have 
different dependencies, both of which have been previously rejected. Therefore, prior 
art applies. 
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Conclusion 

44. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

45. This action is a final rejection and is intended to close the prosecution of this 
application. Applicant's reply under 37 CFR 1.113 to this action is limited either to an 
appeal to the Board of Patent Appeals and Interferences or to an amendment complying 
with the requirements set forth below. 

If applicant should desire to appeal any rejection made by the examiner, a Notice 
of Appeal must be filed within the period for reply identifying the rejected claim or claims 
appealed. The Notice of Appeal must be accompanied by the required appeal fee. 
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If applicant should desire to file an amendment, entry of a proposed amendment 
after final rejection cannot be made as a matter of right unless it merely cancels claims 
or complies with a formal requirement made earlier. Amendments touching the merits 
of the application which otherwise might not be proper may be admitted upon a showing 
a good and sufficient reasons why they are necessary and why they were not presented 
earlier. If the applicant should still desire to file an amendment and does not meet the 
above requirements, the applicant has the option to file for a request for continued 
examination. 

A reply under 37 CFR 1.1 13 to a final rejection must include the appeal from, or 
cancellation of, each rejected claim. The filing of an amendment after final rejection, 
whether or not it is entered, does not stop the running of the statutory period for reply to 
the final rejection unless the examiner holds the claims to be in condition for allowance. 
Accordingly, if a Notice of Appeal has not been filed properly within the period for reply, 
or any extension of this period obtained under either 37 CFR 1.136(a) or (b), the 
application will become abandoned. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kathleen S. Yuan whose telephone number is (571)272- 
2902. The examiner can normally be reached on Monday to Thursdays, 9 AM to 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Mancuso can be reached on (571)272-7695. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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